How are evidence and knowledge used in orthopaedic decision-making? Three comparative case studies of different approaches to implementation of clinical guidance in practice

Background The uptake and use of clinical guidelines is often insufficient to change clinical behaviour and reduce variation in practice. As a consequence of diverse organisational contexts, the simple provision of guidelines cannot ensure fidelity or guarantee their use when making decisions. Implementation research in surgery has focused on understanding what evidence exists for clinical practice decisions but limits understanding to the technical, educational and accessibility issues. This research aims to identify where, when and how evidence and knowledge are used in orthopaedic decision-making and how variation in these factors contributes to different approaches to implementation of clinical guidance in practice. Methods We used in-depth case studies to examine guideline implementation in real-life surgical practice. We conducted comparative case studies in three English National Health Service hospitals over a 12-month period. Each in-depth case study consisted of a mix of qualitative methods including interviews, observations and document analysis. Data included field notes from observations of day-to-day practice, 64 interviews with NHS surgeons and staff and the collection of 121 supplementary documents. Results Case studies identified 17 sources of knowledge and evidence which influenced clinical decisions in elective orthopaedic surgery. A comparative analysis across cases revealed that each hospital had distinct approaches to decision-making. Decision-making is described as occurring as a result of how 17 types of knowledge and evidence were privileged and of how they interacted and changed in context. Guideline implementation was contingent and mediated through four distinct contextual levels. Implementation could be assessed for individual surgeons, groups of surgeons or the organisation as a whole, but it could also differ between these levels. Differences in how evidence and knowledge were used contributed to variations in practice from guidelines. Conclusion A range of complex and competing sources of evidence and knowledge exists which influence the working practices of healthcare professionals. The dynamic selection, combination and use of each type of knowledge and evidence influence the implementation and use of clinical guidance in practice. Clinical guidelines are a fundamental part of practice, but represent only one type of evidence influencing clinical decisions. In the orthopaedic speciality, other distinct sources of evidence and knowledge are selected and used which impact on how guidelines are implemented. New approaches to guideline implementation need to appreciate and incorporate this diverse range of knowledge and evidence which influences clinical decisions and to take account of the changing contexts in which decisions are made

The barriers and facilitators to the implementation of clinical guidance in elective orthopaedic surgery : a qualitative study protocol

Background: Clinical guidelines in orthopaedic surgery aim to improve the efficiency, quality and outcomes of patient care by ensuring that treatment recommendations are based on the best available evidence. The simple provision of guidelines, however, does not ensure fidelity or guarantee their uptake and use in surgical practice. Research exploring the factors that affect surgeons’ use of evidence and guidelines has focused on understanding what evidence exists for current clinical decisions. This narrowed scope emphasises the technical, educational and accessibility issues but overlooks wider factors that help explain how and why guidelines are not implemented and used in surgery. It is also important to understand how we can encourage the implementation processes in practice. By taking a social science perspective to examine orthopaedic surgery, we move beyond the narrow focus and explore how and why clinical guidelines struggle to achieve full uptake. We aim to explore guideline uptake to discover the factors that contribute to, or complicate, appropriate implementation in this field. We need to go beyond traditional views and experimental methods to examine the barriers and facilitators of implementation in real-life NHS surgical practice. These could be multifactorial, linked to individual, organisational or contextual influences, which act on the guideline implementation process. Methods/design: We will use ethnographic methods to conduct case studies in three English NHS hospitals. Within each case, we will conduct observations, interviews and analysis of key documents to understand experiences, complex processes and decisions made and the role of clinical guidance and other sources of evidence within orthopaedic surgery. The data will be transcribed and analysed thematically. Comparisons will be made within cases and across cases. Discussion: Guidelines are a fundamental part of clinical practice, and various factors must be considered when preparing for their successful implementation into organisations. Understanding the views and experiences of a range of surgical, clerical and managerial staff across multiple orthopaedic departments will capture the complexity and variety of factors that can influence surgical decisions. The findings of our study will identify the specific features of orthopaedic practice to help guide the development of strategies to facilitate guideline uptake in everyday surgical work

The Extended Pillar Integration Process (ePIP): A Data Integration Method Allowing the Systematic Synthesis of Findings From Three Different Sources

Mixed methods research requires data integration from multiple sources. Existing techniques are restricted to integrating a maximum of two data sources, do not provide step-by-step guidance or can be cumbersome where many data need to be integrated. We have solved these limitations through the development of the extended Pillar Integration Process (ePIP), a method which contributes to the field of mixed methods by being the first data integration method providing explicit steps on how to integrate data from three data sources. The ePIP provides greater transparency, validity and consistency compared to existing methods. We provide two worked examples from health sciences and automotive human factors, highlighting its value as a mixed methods integration tool

A qualitative exploration of evidence-based decision making in public health practice and policy : the perceived usefulness of a diabetes economic model for decision makers

Purpose The purpose of this paper is to explore the perceived usefulness of a diabetes economic model as a potential tool for aiding evidence-based decision making in public health. Methods Fifteen interviews and two focus groups with four participants each were conducted with health and management professionals working in one public health department in a local council. Data were analysed using inductive thematic analysis to generate four themes. Findings Findings reflect attitudes and beliefs of a diverse staff group situated in public health services. They demonstrate that the model had perceived usefulness, and participants reported positive views regarding the principles of economic modelling for decision making. The model was perceived as useful but potentially problematic in practice due to organisational constraints linked to limited resources, restricted budgets and local priorities. Differences in institutional logics of staff working in public health and local government were identified as a potential barrier to the use of the model in practice. Discussion The findings highlight anticipated challenges that individuals tasked with making decisions for public health practice and policy could face if they selected to implement an economic modelling approach to fulfill their evidence needs. Previous studies have revealed that healthcare decision makers would find evidence around the economic impacts of public health interventions useful, but this information was not always available in the format required. This paper provides insights into how staff working in public health perceive economic modelling, and explores how they consider evidence from a model when making public health practice and policy decisions

Total hip replacement for the treatment of end stage arthritis of the hip : a systematic review and meta-analysis

Background: Evolvements in the design, fixation methods, size, and bearing surface of implants for total hip replacement (THR) have led to a variety of options for healthcare professionals to consider. The need to determine the most optimal combinations of THR implant is warranted. This systematic review evaluated the clinical effectiveness of different types of THR used for the treatment of end stage arthritis of the hip. Methods: A comprehensive literature search was undertaken in major health databases. Randomised controlled trials (RCTs) and systematic reviews published from 2008 onwards comparing different types of primary THR in patients with end stage arthritis of the hip were included. Results: Fourteen RCTs and five systematic reviews were included. Patients experienced significant post-THR improvements in Harris Hip scores, but this did not differ between impact types. There was a reduced risk of implant dislocation after receiving a larger femoral head size (36 mm vs. 28 mm; RR = 0.17, 95% CI: 0.04, 0.78) or cemented cup (vs. cementless cup; pooled odds ratio: 0.34, 95% CI: 0.13, 0.89). Recipients of cross-linked vs. conventional polyethylene cup liners experienced reduced femoral head penetration and revision. There was no impact of femoral stem fixation and cup shell design on implant survival rates. Evidence on mortality and complications (aseptic loosening, femoral fracture) was inconclusive. Conclusions: The majority of evidence was inconclusive due to poor reporting, missing data, or uncertainty in treatment estimates. The findings warrant cautious interpretation given the risk of bias (blinding, attrition), methodological limitations (small sample size, low event counts, short follow-up), and poor reporting. Long-term pragmatic RCTs are needed to allow for more definitive conclusions. Authors are encouraged to specify the minimal clinically important difference and power calculation for their primary outcome(s) as well CONSORT, PRISMA and STROBE guidelines to ensure better reporting and more reliable production and assessment of evidence

Evidence-based commissioning in the English NHS : who uses which sources of evidence? A survey 2010/2011

Objectives: To investigate types of evidence used by healthcare commissioners when making decisions and whether decisions were influenced by commissioners’ experience, personal characteristics or role at work. Design: Cross-sectional survey of 345 National Health Service (NHS) staff members. Setting: The study was conducted across 11 English Primary Care Trusts between 2010 and 2011. Participants: A total of 440 staff involved in commissioning decisions and employed at NHS band 7 or above were invited to participate in the study. Of those, 345 (78%) completed all or a part of the survey. Main outcome measures: Participants were asked to rate how important different sources of evidence (empirical or practical) were in a recent decision that had been made. Backwards stepwise logistic regression analyses were undertaken to assess the contributions of age, gender and professional background, as well as the years of experience in NHS commissioning, pay grade and work role. Results: The extent to which empirical evidence was used for commissioning decisions in the NHS varied according to the professional background. Only 50% of respondents stated that clinical guidelines and cost-effectiveness evidence were important for healthcare decisions. Respondents were more likely to report use of empirical evidence if they worked in Public Health in comparison to other departments (p<0.0005, commissioning and contracts OR 0.32, 95%CI 0.18 to 0.57, finance OR 0.19, 95%CI 0.05 to 0.78, other departments OR 0.35, 95%CI 0.17 to 0.71) or if they were female (OR 1.8 95% CI 1.01 to 3.1) rather than male. Respondents were more likely to report use of practical evidence if they were more senior within the organisation (pay grade 8b or higher OR 2.7, 95%CI 1.4 to 5.3, p=0.004 in comparison to lower pay grades). Conclusions: Those trained in Public Health appeared more likely to use external empirical evidence while those at higher pay scales were more likely to use practical evidence when making commissioning decisions. Clearly, National Institute for Clinical Excellence (NICE) guidance and government publications (eg, National Service Frameworks) are important for decision-making, but practical sources of evidence such as local intelligence, benchmarking data and expert advice are also influential

Knowledge mobilisation in orthopaedic surgery in England : why hierarchies of knowledge bear little relation to the hierarchy of evidence in professionally socialised groups

Background: Healthcare policy encourages the use of scientific evidence in the delivery of healthcare services. However, the complexity of practice restricts the use of codified knowledge in clinical guidelines despite continued effort from policymakers to ensure their use in practice. This study adopts a knowledge mobilisation perspective to explore the multiple levels through which scientific evidence is enacted to generate variation in practice. Aim: To explore how professionalised groups of clinicians mobilised knowledge in the highly-professionalised and organisationally-regulated context of orthopaedic surgery. Methods: Three contrasting NHS hospitals in England were examined using case study methods from 2014–15. Data included 64 interviews with surgeons and NHS staff, nine months of observation of day-to-day practice, and collection of 121 supplementary documents. A multilevel thematic analysis and cross-case comparison explored how individual surgeons, groups of professionals and hospital organisations mobilised knowledge. Findings: The findings described three themes to illustrate how variation in orthopaedic practice emerged: 1) professional identities; 2) knowledge acquisition; and 3) the contextual contingencies of practice. The professional groups which surgeons identified with had significant influence on how knowledge was mobilised within the organisations. Conclusions: Knowledge owned by professionally socialised surgical groups was central to explaining variation observed in the delivery of healthcare services. Hierarchies of knowledge in the practice of orthopaedic surgery bore little relation to the hierarchy of evidence which is foundational to the production of clinical guidance and guidelines. Knowledge defined and privileged within professional surgeon groups carried significant weight in practice, and generated contingent knowledge mobilisation

Belimumab : a technological advance for systemic lupus erythematosus patients? Report of a systematic review and meta-analysis

Objectives: To undertake a systematic review and meta-analysis to investigate clinical effectiveness of belimumab for patients with systemic lupus erythematosus (SLE) and antinuclear and/or anti-double-stranded DNA (dsDNA) autoantibodies. Methods: We searched eight electronic databases and reference lists for randomised controlled trials (RCTs) of belimumab against placebo or best supportive care. Quality assessment and random effects meta-analysis were undertaken. Design: A meta-analysis of RCTs. Participants: 2133 SLE patients. Primary and secondary outcome measures: SLE Responder Index (SRI) at week 52. Results: Three double-blind placebo-controlled RCTs (L02, BLISS-52 BLISS-76) investigated 2133 SLE patients. BLISS-52 and BLISS-76 trials recruited patients with antinuclear and/or anti-dsDNA autoantibodies and demonstrated belimumab effectiveness for the SRI at week 52. Ethnicity and geographical location of participants varied considerably between BLISS trials. Although tests for statistical heterogeneity were negative, BLISS-52 results were systematically more favourable for all measured outcomes. Meta-analysis of pooled 52-week SRI BLISS results showed benefit for belimumab (OR 1.63, 95% CI 1.27 to 2.09). By week 76, the primary SRI outcome in BLISS-76 was not statistically significant (OR 1.31, 95% CI 0.919 to 1.855)

Aspirin for prophylactic use in the primary prevention of cardiovascular disease and cancer : a systematic review and overview of reviews

Background: Prophylactic aspirin has been considered to be beneficial in reducing the risks of heart disease and cancer. However, potential benefits must be balanced against the possible harm from side effects, such as bleeding and gastrointestinal (GI) symptoms. It is particularly important to know the risk of side effects when aspirin is used as primary prevention - that is when used by people as yet free of, but at risk of developing, cardiovascular disease (CVD) or cancer. In this report we aim to identify and re-analyse randomised controlled trials (RCTs), systematic reviews and meta-analyses to summarise the current scientific evidence with a focus on possible harms of prophylactic aspirin in primary prevention of CVD and cancer. Objectives: To identify RCTs, systematic reviews and meta-analyses of RCTs of the prophylactic use of aspirin in primary prevention of CVD or cancer. To undertake a quality assessment of identified systematic reviews and meta-analyses using meta-analysis to investigate study-level effects on estimates of benefits and risks of adverse events; cumulative meta-analysis; exploratory multivariable meta-regression; and to quantify relative and absolute risks and benefits. Methods: We identified RCTs, meta-analyses and systematic reviews, and searched electronic bibliographic databases (from 2008 September 2012) including MEDLINE, Cochrane Central Register of Controlled Trials, Database of Abstracts of Reviews of Effects, NHS Centre for Reviews and Dissemination, and Science Citation Index. We limited searches to publications since 2008, based on timing of the most recent comprehensive systematic reviews. Results: In total, 2572 potentially relevant papers were identified and 27 met the inclusion criteria. Benefits of aspirin ranged from 6% reduction in relative risk (RR) for all-cause mortality [RR 0.94, 95% confidence interval (CI) 0.88 to 1.00] and 10% reduction in major cardiovascular events (MCEs) (RR 0.90, 95% CI 0.85 to 0.96) to a reduction in total coronary heart disease (CHD) of 15% (RR 0.85, 95% CI 0.69 to 1.06). Reported pooled odds ratios (ORs) for total cancer mortality ranged between 0.76 (95% CI 0.66 to 0.88) and 0.93 (95% CI 0.84 to 1.03). Inclusion of the Women's Health Study changed the estimated OR to 0.82 (95% CI 0.69 to 0.97). Aspirin reduced reported colorectal cancer (CRC) incidence (OR 0.66, 95% CI 0.90 to 1.02). However, including studies in which aspirin was given every other day raised the OR to 0.91 (95% CI 0.74 to 1.11). Reported cancer benefits appeared approximately 5 years from start of treatment. Calculation of absolute effects per 100,000 patient-years of follow-up showed reductions ranging from 33 to 46 deaths (all-cause mortality), 60-84 MCEs and 47-64 incidents of CHD and a possible avoidance of 34 deaths from CRC. Reported increased RRs of adverse events from aspirin use were 37% for GI bleeding (RR 1.37, 95% CI 1.15 to 1.62), between 54% (RR 1.54, 95% CI 1.30 to 1.82) and 62% (RR 1.62, 95% CI 1.31 to 2.00) for major bleeds, and between 32% (RR 1.32, 95% CI 1.00 to 1.74) and 38% (RR 1.38, 95% CI 1.01 to 1.82) for haemorrhagic stroke. Pooled estimates of increased RR for bleeding remained stable across trials conducted over several decades. Estimates of absolute rates of harm from aspirin use, per 100,000 patient-years of follow-up, were 99-178 for non-trivial bleeds, 46-49 for major bleeds, 68-117 for GI bleeds and 8-10 for haemorrhagic stroke. Meta-analyses aimed at judging risk of bleed according to sex and in individuals with diabetes were insufficiently powered for firm conclusions to be drawn. Limitations: Searches were date limited to 2008 because of the intense interest that this subject has generated and the cataloguing of all primary research in so many previous systematic reviews. A further limitation was our potential over-reliance on study-level systematic reviews in which the person-years of follow-up were not accurately ascertainable. However, estimates of number of events averted or incurred through aspirin use calculated from data in study-level meta-analyses did not differ substantially from estimates based on individual patient data-level meta-analyses, for which person-years of follow-up were more accurate (although based on less-than-complete assemblies of currently available primary studies). Conclusions: We have found that there is a fine balance between benefits and risks from regular aspirin use in primary prevention of CVD. Effects on cancer prevention have a long lead time and are at present reliant on post hoc analyses. All absolute effects are relatively small compared with the burden of these diseases. Several potentially relevant ongoing trials will be completed between 2013 and 2019, which may clarify the extent of benefit of aspirin in reducing cancer incidence and mortality. Future research considerations include expanding the use of IPD meta-analysis of RCTs by pooling data from available studies and investigating the impact of different dose regimens on cardiovascular and cancer outcomes

Aspirin in primary prevention of cardiovascular disease and cancer : a systematic review of the balance of evidence from reviews of randomized trials

Background: Aspirin has been recommended for primary prevention of cardiovascular disease (CVD) and cancer, but overall benefits are unclear. We aimed to use novel methods to re-evaluate the balance of benefits and harms of aspirin using evidence from randomised controlled trials, systematic reviews and meta-analyses. Methods and Findings: Data sources included ten electronic bibliographic databases, contact with experts, and scrutiny of reference lists of included studies. Searches were undertaken in September 2012 and restricted to publications since 2008. Of 2,572 potentially relevant papers 27 met the inclusion criteria. Meta-analysis of control arms to estimate event rates, modelling of all-cause mortality and L'Abbé plots to estimate heterogeneity were undertaken. Absolute benefits and harms were low: 60-84 major CVD events and 34-36 colorectal cancer deaths per 100,000 person-years were averted, whereas 46-49 major bleeds and 68-117 gastrointestinal bleeds were incurred. Reductions in all-cause mortality were minor and uncertain (Hazard Ratio 0.96; 95% CI: 0.90-1.02 at 20 years, Relative Risk [RR] 0.94, 95% CI: 0.88-1.00 at 8 years); there was a non-significant change in total CVD (RR 0.85, 95% CI: 0.69-1.06) and change in total cancer mortality ranged from 0.76 (95% CI: 0.66-0.88) to 0.93 (95% CI: 0.84-1.03) depending on follow-up time and studies included. Risks were increased by 37% for gastrointestinal bleeds (RR 1.37, 95% CI: 1.15-1.62), 54%-66% for major bleeds (Rate Ratio from IPD analysis 1.54, 95% CI: 1.30-1.82, and RR 1.62, 95% CI: 1.31-2.00), and 32%-38% for haemorrhagic stroke (Rate Ratio from IPD analysis 1.32; 95% CI: 1.00-1.74; RR 1.38; 95% CI: 1.01-1.82). Conclusions: Findings indicate small absolute effects of aspirin relative to the burden of these diseases. When aspirin is used for primary prevention of CVD the absolute harms exceed the benefits. Estimates of cancer benefit rely on selective retrospective re-analysis of RCTs and more information is needed